{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Debug: Miscellaneous\n",
    "In this example notebook, we will give few examples to help users to debug in different scenarios.  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How to save a folder from TI EdgeAI cloud, so it can be shared\n",
    "Say we want to share custom-artifacts via E2E for support. First we compress the file and then we download it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import FileLink\n",
    "!tar chvfz custom-artifacts.tar.gz custom-artifacts\n",
    "FileLink(\"custom-artifacts.tar.gz\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How to run and compare a model compiled with different compilation option \n",
    "Say we compiled and run an 8-bit model using custom-models notebooks, but accuracy is not good enough and we would like to test other configuration. \n",
    "Below cell showcase an output of a model compiled for 8 bit and accuracy level of 0, which shows a low accuracy output for classification (image is wrongly classified). Below code also compile and run the same model at 16 bits with accuracy level 1, these latest compilation options gives a better accuracy at the expense of performance (lower fps).\n",
    "Users can use 16 bits + accuracy level of 1 as their accuracy bar, and by changing other configuration params (for example increasing number of calibration frames and/or iterations) they can increase 8 bits accuracy to get closer to 16 bits.\n",
    "\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "<b>Note:</b> Custom-model notebooks won't run on EVM only approach. Please use TI EdgeAI Cloud for run them.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import papermill as pm\n",
    "import subprocess\n",
    "\n",
    "def generate_html(notebook):\n",
    "    generate = subprocess.run(\n",
    "        [\n",
    "            \"jupyter\",\n",
    "            \"nbconvert\",\n",
    "            notebook,\n",
    "            \"--to=html\",\n",
    "        ]\n",
    "    )\n",
    "    print(\"HTML created\")\n",
    "    return True\n",
    "\n",
    "# We purposely will build a model with low accuracy by selecting in TIDL compilation params 8-bits\n",
    "# and accuracy level 0 \n",
    "pm.execute_notebook(\n",
    "   'custom-model-tfl.ipynb',\n",
    "   'custom-model-tfl_8bits.ipynb',\n",
    "   parameters = dict(num_bits=8, accuracy=0)\n",
    ")\n",
    "generate_html(\"custom-model-tfl_8bits.ipynb\")\n",
    "\n",
    "# We purposely will build a model with high accuracy (lower performance) by selecting in TIDL compilation\n",
    "# params 16-bits and accuracy level 1 \n",
    "pm.execute_notebook(\n",
    "   'custom-model-tfl.ipynb',\n",
    "   'custom-model-tfl_16bits.ipynb',\n",
    "   parameters = dict(num_bits=16, accuracy=1)\n",
    ")\n",
    "generate_html(\"custom-model-tfl_16bits.ipynb\")\n",
    "\n",
    "try:\n",
    "    os.remove(\"custom-model-tfl_16bits.ipynb\")\n",
    "    os.remove(\"custom-model-tfl_8bits.ipynb\")\n",
    "except:\n",
    "  print(\"Papermill output files not found\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How to compare ARM only vs. Heterogenous execution performance \n",
    "Say we want to compare fps achieved when running your model on Cortex A** core vs. Running it with TIDL subgraph(s) offload. For this purpose, we provided two notebooks examples (vcls-tfl-arm.ipynb and vcls-onnx-arm.ipynb) which run a model in ARM only followed by running the same model with TIDL subgraph offload. Below cells will run those notebooks and generate an output report as an example.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import papermill as pm\n",
    "import subprocess\n",
    "\n",
    "def generate_html(notebook):\n",
    "    generate = subprocess.run(\n",
    "        [\n",
    "            \"jupyter\",\n",
    "            \"nbconvert\",\n",
    "            notebook,\n",
    "            \"--to=html\",\n",
    "        ]\n",
    "    )\n",
    "    print(\"HTML created\")\n",
    "    return True\n",
    "\n",
    "pm.execute_notebook(\n",
    "   'vcls-tfl-arm.ipynb',\n",
    "   'vcls-tfl-arm_output.ipynb',\n",
    ")\n",
    "generate_html(\"vcls-tfl-arm_output.ipynb\")\n",
    "\n",
    "pm.execute_notebook(\n",
    "   'vcls-onnx-arm.ipynb',\n",
    "   'vcls-onnx-arm_output.ipynb',\n",
    ")\n",
    "\n",
    "generate_html(\"vcls-onnx-arm_output.ipynb\")\n",
    "\n",
    "try:\n",
    "    os.remove(\"vcls-tfl-arm_output.ipynb\")\n",
    "    os.remove(\"vcls-onnx-arm_output.ipynb\")\n",
    "except:\n",
    "  print(\"Papermill output files not found\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
